Derivatives are essential in a variety of fields and applications. By way of example, one such field is computer graphics, where derivatives are used for global illumination optimization, dynamics problems, and comparing surface normals and curvature. In general, derivatives may be computed either manually or automatically. Manual differentiation is so tedious and error prone that it only is used for the simplest of problems. In the majority of cases, the derivative is computed using automatic techniques running on a computing device. There are at least three automatic techniques for computing derivatives, including finite differencing, automatic differentiation, and symbolic differentiation.
The finite difference technique approximates the derivative by computing differences. However, because it is so inaccurate and inefficient as compared to other techniques, it is used rarely. Automatic differentiation is a technique for evaluating a function and its derivative simultaneously. There are two basic forms of automatic differentiation: forward and reverse. Reverse automatic differentiation works well on functions that have multiple inputs and only one output. In this situation, however, forward automatic differentiation is inefficient. The inverse is also true: for functions having one input and multiple outputs, forward automatic differentiation is efficient but reverse automatic differentiation is inefficient. In order to avoid inefficient automatic differentiation, depending on the type of function a user is required to specify either forward or reverse automatic differentiation. This requirement increases the complexity and decreases the speed of the automatic differentiation process. In addition, neither forward nor reverse automatic differentiation works efficiently for functions having multiple inputs and multiple outputs.
Symbolic differentiation has traditionally been the domain of expensive and proprietary math systems (such as Mathematica®). These systems can compute efficient derivatives for simple functions. However, as the size and complexity of the function increases so does the computation time and memory necessary to compute its derivative. This means that for some functions using a symbolic differentiation technique causes the computing device to exhausts its memory (meaning that the derivative cannot be computed at all) or that the symbolic differentiation technique takes an inordinate amount of time to compute the derivative.